add VS code system tests and manual test plans#19208
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds system tests and a manual testing plan for VS Code (and VSCodium derivatives) to expand NVDA's test coverage for applications available in GitHub Actions runners. The PR introduces automated tests for basic VS Code functionality and provides a comprehensive manual smoke testing guide for IDE accessibility verification.
- Adds 4 new automated test functions for VS Code (sidebar toggle, command palette, file navigation, and search panel)
- Creates a comprehensive manual testing plan covering 6 key areas of VS Code accessibility
- Updates copyright headers and imports to support the new test infrastructure
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 8 comments.
| File | Description |
|---|---|
| tests/system/robot/vscodeTests.py | Adds 4 new automated test functions for VS Code features; renames existing function and updates imports with type hints |
| tests/manual/applications/IDEs/vsCodium.md | Creates new manual smoke testing plan covering application launch, file operations, editor accessibility, command palette, extensions, and terminal |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 19 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
SaschaCowley
left a comment
SaschaCowley
left a comment
There was a problem hiding this comment.
This looks fine to me, though I don't understand why we're adding accessibility tests for Codium's UI. Most, if not all, of these tests are based on Codium's implementation, not on our ARIA/Electron support. They're also going to be quite brittle and prone to break with Codeium updates.
Link to issue number:
Part of #18762
Split from #18419
Summary of the issue:
We should expand our system tests to test against more applications.
Starting with open source applications that are easily available within GitHub actions runners,
Description of user facing changes:
None
Description of developer facing changes:
None
Description of development approach:
Perform manual tests and implement equivalent system tests where possible
Testing strategy:
See changes
Known issues with pull request:
Code Review Checklist: